Search for: All records

The small sizes of most marine plankton necessitate that plankton sampling occur on fine spatial scales, yet our questions often span large spatial areas. Underwater imaging can provide a solution to this sampling conundrum but collects large quantities of data that require an automated approach to image analysis. Machine learning for plankton classification, and high-performance computing (HPC) infrastructure, are critical to rapid image processing; however, these assets, especially HPC infrastructure, are only available post-cruise leading to an ‘after-the-fact’ view of plankton community structure. To be responsive to the often-ephemeral nature of oceanographic features and species assemblages in highly dynamic current systems, real-time data are key for adaptive oceanographic sampling. Here we used the new In-situ Ichthyoplankton Imaging System-3 (ISIIS-3) in the Northern California Current (NCC) in conjunction with an edge server to classify imaged plankton in real-time into 170 classes. This capability together with data visualization in a heavy.ai dashboard makes adaptive real-time decision-making and sampling at sea possible. Dual ISIIS-Deep-focus Particle Imager (DPI) cameras sample 180 L s -1 , leading to >10 GB of video per min. Imaged organisms are in the size range of 250 µm to 15 cm and include abundant crustaceans, fragile taxa (e.g., hydromedusae, salps), faster swimmers (e.g., krill), and rarer taxa (e.g., larval fishes). A deep learning pipeline deployed on the edge server used multithreaded CPU-based segmentation and GPU-based classification to process the imagery. AVI videos contain 50 sec of data and can contain between 23,000 - 225,000 particle and plankton segments. Processing one AVI through segmentation and classification takes on average 3.75 mins, depending on biological productivity. A heavyDB database monitors for newly processed data and is linked to a heavy.ai dashboard for interactive data visualization. We describe several examples where imaging, AI, and data visualization enable adaptive sampling that can have a transformative effect on oceanography. We envision AI-enabled adaptive sampling to have a high impact on our ability to resolve biological responses to important oceanographic features in the NCC, such as oxygen minimum zones, or harmful algal bloom thin layers, which affect the health of the ecosystem, fisheries, and local communities. 

As the basis of oceanic food webs and a key component of the biological carbon pump, planktonic organisms play major roles in the oceans. Their study benefited from the development of in situ imaging instruments, which provide higher spatio-temporal resolution than previous tools. But these instruments collect huge quantities of images, the vast majority of which are of marine snow particles or imaging artifacts. Among them, the In Situ Ichthyoplankton Imaging System (ISIIS) samples the largest water volumes (> 100 L s -1 ) and thus produces particularly large datasets. To extract manageable amounts of ecological information from in situ images, we propose to focus on planktonic organisms early in the data processing pipeline: at the segmentation stage. We compared three segmentation methods, particularly for smaller targets, in which plankton represents less than 1% of the objects: (i) a traditional thresholding over the background, (ii) an object detector based on maximally stable extremal regions (MSER), and (iii) a content-aware object detector, based on a Convolutional Neural Network (CNN). These methods were assessed on a subset of ISIIS data collected in the Mediterranean Sea, from which a ground truth dataset of > 3,000 manually delineated organisms is extracted. The naive thresholding method captured 97.3% of those but produced ~340,000 segments, 99.1% of which were therefore not plankton (i.e. recall = 97.3%, precision = 0.9%). Combining thresholding with a CNN missed a few more planktonic organisms (recall = 91.8%) but the number of segments decreased 18-fold (precision increased to 16.3%). The MSER detector produced four times fewer segments than thresholding (precision = 3.5%), missed more organisms (recall = 85.4%), but was considerably faster. Because naive thresholding produces ~525,000 objects from 1 minute of ISIIS deployment, the more advanced segmentation methods significantly improve ISIIS data handling and ease the subsequent taxonomic classification of segmented objects. The cost in terms of recall is limited, particularly for the CNN object detector. These approaches are now standard in computer vision and could be applicable to other plankton imaging devices, the majority of which pose a data management problem. 

Microbes and sunlight convert terrigenous dissolved organic matter (DOM) in surface waters to greenhouse gases. Prior studies show contrasting results about how biological and photochemical processes interact to contribute to the degradation of DOM. In this study, DOM leached from the organic layer of tundra soil was exposed to natural sunlight or kept in the dark, incubated in the dark with the natural microbial community, and analyzed for gene expression and DOM chemical composition. Microbial gene expression (metatranscriptomics) in light and dark treatments diverged substantially after 4 hours. Gene expression suggested that sunlight exposure of DOM initially stimulated microbial growth by (a) replacing the function of enzymes that degrade higher molecular weight DOM such as enzymes for aromatic carbon degradation, oxygenation, and decarboxylation, and (b) releasing low molecular weight compounds and inorganic nutrients from DOM. However, growth stimulation following sunlight exposure of DOM came at a cost. Sunlight depleted the pool of aromatic compounds that supported microbial growth in the dark treatment, ultimately causing slower growth in the light treatment over 5 days. These first measurements of microbial metatranscriptomic responses to photo-alteration of DOM provide a mechanistic explanation for how sunlight exposure of terrigenous DOM alters microbial processing and respiration of DOM. 

High‐throughput phenotyping systems are powerful, dramatically changing our ability to document, measure, and detect biological phenomena. Here, we describe a cost‐effective combination of a custom‐built imaging platform and deep‐learning‐based computer vision pipeline. A minimal version of the maize (Zea mays) ear scanner was built with low‐cost and readily available parts. The scanner rotates a maize ear while a digital camera captures a video of the surface of the ear, which is then digitally flattened into a two‐dimensional projection. Segregating GFP and anthocyanin kernel phenotypes are clearly distinguishable in ear projections and can be manually annotated and analyzed using image analysis software. Increased throughput was attained by designing and implementing an automated kernel counting system using transfer learning and a deep learning object detection model. The computer vision model was able to rapidly assess over 390 000 kernels, identifying male‐specific transmission defects across a wide range of GFP‐marked mutant alleles. This includes a previously undescribed defect putatively associated with mutation of Zm00001d002824, a gene predicted to encode a vacuolar processing enzyme. Thus, by using this system, the quantification of transmission data and other ear and kernel phenotypes can be accelerated and scaled to generate large datasets for robust analyses.

 

Microbes and sunlight convert terrigenous dissolved organic matter (DOM) in surface waters to greenhouse gases. Prior studies show contrasting results about how biological and photochemical processes interact to contribute to the degradation of DOM. In this study, DOM leached from the organic layer of tundra soil was exposed to natural sunlight or kept in the dark, incubated in the dark with the natural microbial community, and analysed for gene expression and DOM chemical composition. Microbial gene expression (metatranscriptomics) in light and dark treatments diverged substantially after 4 h. Gene expression suggested that sunlight exposure of DOM initially stimulated microbial growth by (i) replacing the function of enzymes that degrade higher molecular weight DOM such as enzymes for aromatic carbon degradation, oxygenation, and decarboxylation, and (ii) releasing low molecular weight compounds and inorganic nutrients from DOM. However, growth stimulation following sunlight exposure of DOM came at a cost. Sunlight depleted the pool of aromatic compounds that supported microbial growth in the dark treatment, ultimately causing slower growth in the light treatment over 5 days. These first measurements of microbial metatranscriptomic responses to photo‐alteration of DOM provide a mechanistic explanation for how sunlight exposure of terrigenous DOM alters microbial processing and respiration of DOM.